1. Field of the Invention
This invention generally relates to data networking technologies and more particularly, to a method and apparatus for multiplexing bytes over parallel communication links using data slices.
2. Description of the Related Art
Increasing reliability and availability of high speed networks has fueled the growth of many new telecommunication based services. The Internet is one widely known and widely used network being used to deliver electronic commerce (e-commerce), telecommuting, and interactive entertainment services around the world. Predicted growth in these commercial endeavors will continue to far out pace the availability of bandwidth current telecommunication vendors can provide.
Telecommunication systems used on the Internet and other worldwide networks typically include local area networks coupled to very high speed wide area networks or back bones. The local area networks (LAN) are installed at small campuses or businesses and typically low cost and provide bandwidth capacity from 10 Mbps to 100 Mbps.
In contrast, WAN (wide area networks) generally cost more to implement and provide higher bandwidth capacities. WANs generally operate at much wider range of bandwidth capacities ranging from tens of kilobits to gigabits per second. Future networks will likely operate in the terabit range or higher. Further, WANs must transport variable size packets generated by different networks having different bandwidth characteristics. These packets frequently travel distances spanning numerous continents. Consequently, the right-of-ways for land based connections and frequency licenses used in wireless connections can make WANs costly to implement.
Synchronous digital hierarchy (SDH) is a protocol established to carry the needs of many different types of communication systems including voice, data, and video. Three different versions of SDH exist: SDH-Europe, SDH-Japan, and SONET for North America. Except for some minor differences between these three versions, these systems are essentially compatible. Collectively, SDH will be referred to as SONET.
SONET is designed to accommodate a wide mixture of protocols and bandwidths such as T-1, T-3, E-1 and other high speed protocols. Network systems implementing SONET are capable of stripping bandwidth off at geographically distant locations with little difficulty. Unlike other WAN protocols, the SONET design allows high speed data to be stripped off at distant locations without demultiplexing and reagreggating bandwidth at each drop point. Instead of conventional multiplexers, the SONET system uses add-drop multiplexers (ADM) to distribute high speed data at various geographic locations. For at least these reasons, SONET is especially desirable in video systems, interactive gaming, e-commerce, and other high bandwidth low-latency applications spread over large geographic areas.
High speed SONET currently available operates at rates up to) approximately 10-Gbps per second and is otherwise known as OC-192. Essentially OC-192 is 192 times faster than OC-1 (51.85 Mbps). All SONET and SDH systems are designed to operate at multiples of 51.85 Mbps to allow for efficient conversion from one data rate to the other.
In practice OC-192 is difficult to implement over most networks. Specifically, OC-192 does not work well over older transmission mediums which may have geometric irregularities or impurities in the transmission mediums composition. For example, a phenomenon known as polarization-mode dispersion can cause a signal frequency to shift over long distances and introduce noise and distortion on an OC-192 communication link. Even with new cables having few irregularities, OC-192 may be difficult to implement without developing expensive optical transmitters operating at very high frequencies. These high speed transmitter devices for OC-192 can be extremely difficult to design and prone to failure.
Many WANs have achieved the high speed bandwidth at OC-192 and higher by aggregating multiple lower speed optical or copper channels. Numerous OC-48 channels have been successfully combined together using a technology known as wave division multiplexing or WDM.
On a fiber optic network, WDM takes advantage of the inherent high bandwidth capacity of an optical fiber by transmitting data in parallel over the optical medium. Signals co-exist on the optical medium by transmitting data with lasers having different wave lengths. Each wave length can be used to establish a separate sublink between the transmitter-receiver pair. The system receiving the WDM transmission includes optical receivers sensitive to the different wave lengths or frequencies used during the transmission. By transmitting information in parallel, overall capacity on a SONET system can be increased by the number sublinks used in the transmission. WDM has rapidly increased in popularity because it allows for high speed transmission at a lower cost and a higher degree of reliability. Further, data transmission occurs over a series of slower links, which are less expensive to create and are more robust in less than ideal communication environments.
In practice, WDM works well in applications that access the multiple sublinks in parallel. However, WDM does not work well when using network interconnect devices such as routers, switches and hubs which are better suited for use with a single sublink. These network interconnect devices typically transmit information over a single sublink between any two devices. Clearly, the challenge in using WDM with conventional network interconnect devices such as routers, switches and hubs, lies in aggregating the bandwidth from several parallel links into a single channel.
Packet-by-packet striping is one method of utilizing multiple parallel sublinks to carry data from a single communication link. Packet-by-striping distributes one packet on a first sublink and subsequent packets on subsequent sublinks. This technique evenly distributes multiple packets over multiple links and transmits the data in parallel. Unfortunately, packet-by-packet striping has limitations if one is interested in keeping the packets in order and processing them in a work conserving scheme.
In a work conserving queuing scheme, servers and networks should not be idle when packets in the queue are ready for transmission. For example, a conventional network using WDM may extract packets out of order and send the packets over a communication link which happens to be idle. This technique is work conserving but delivers packets out of order and introduces additional delay reordering packets at the receiver.
Further, packets transmitted out of order in a packet-by-packet striping scheme may require that additional sequence numbers are inserted into each packet. If the sequence number range is large, the packet sizes may be significantly enlarged to hold the sequence number values. This can contribute to increased buffer sizes and other resource utilization.
Conversely, systems which preserve packet order may rot be work conserving with respect to network bandwidth. For example, conventional systems designed to preserve packet ordering may temporarily hold packets in a queue waiting for the next sequential packet to arrive. Meanwhile, buffers and other resources are underutilized waiting for the next sequential packet to arrive.
It is desirable to develop a technique for aggregating multiple high speed links into a single higher speed link for delivery to interconnect devices or other communication points. The technique should be work conserving and also deliver packets in order to reduce processing time associated with reordering packets. This will take advantage of parallel communication technologies such as WDM and facilitate their integration in networks which require a single communication link. A single high speed communication link delivered from multiple parallel sublinks provided over a WAN will enable many more systems to communicate at lower costs and higher efficiencies.